Towards a Scalable and Robust Entity Resolution -Approximate Blocking with Semantic Constraints

Entity resolution, or record linkage, is the process that identifies data records over one or more datasets which refer to the same real world entity. To deal with large datasets, many real-life applications require scalable and high-quality entity resolution techniques. Blocking techniques can help to scale-up the entity resolution process. Locality sensitive hashing (LSH) is an approximate blocking approach that hashes objects within a certain distance into the same block with high probability. This technique can filter out records with low similarities, thus decreases the number of comparisons. However, the traditional approach only considers the textual or string similarity of records while the semantic similarity or constraints of records are ignored. This project is to propose and implement a framework that incorporates semantic constraints into the approximate blocking process to achieve scalable, high performance entity resolution. Firstly, minhashing based locality sensitive hashing methods are applied to generate minhash signatures based on the textual similarity of records. Then, for the semantic constraints, the whole domain knowledge of a dataset is extracted into a domain tree. After applying constraints functions according to a set of pre-set rules, a set of semantic signatures are generated. Then these two sets of signatures are combined to group the records into blocks. The experiments are conducted based on the Cora dataset. The results show that this framework makes blocking much more accurate, and in the meanwhile keeps high completeness.